How to limit the burden of data collection for Quality Indicators based on medical records? The COMPAQH experience

BACKGROUND Our objective was to limit the burden of data collection for Quality Indicators (QIs) based on medical records. METHODS The study was supervised by the COMPAQH project. Four QIs based on medical records were tested: medical record conformity; traceability of pain assessment; screening for nutritional disorders; time elapsed before sending copy of discharge letter to the general practitioner. Data were collected by 6 Clinical Research Assistants (CRAs) in a panel of 36 volunteer hospitals and analyzed by COMPAQH. To limit the burden of data collection, we used the same sample of medical records for all 4 QIs, limited sample size to 80 medical records, and built a composite score of only 10 items to assess medical record completeness. We assessed QI feasibility by completing a grid of 19 potential problems and evaluating time spent. We assessed reliability (kappa coefficient) as well as internal consistency (Cronbach alpha coefficient) in an inter-observer study, and discriminatory power by analysing QI variability among hospitals. RESULTS Overall, 23 115 data items were collected for the 4 QIs and analyzed. The average time spent on data collection was 8.5 days per hospital. The most common feasibility problem was misunderstanding of the item by hospital staff. QI reliability was good (kappa: 0.59-0.97 according to QI). The hospitals differed widely in their ability to meet the quality criteria (mean value: 19-85%). CONCLUSION These 4 QIs based on medical records can be used to compare the quality of record keeping among hospitals while limiting the burden of data collection, and can therefore be used for benchmarking purposes. The French National Health Directorate has included them in the new 2009 version of the accreditation procedure for healthcare organizations.